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Abstract — This paper is focused on the performance analysis 
of binary linear block codes (or ensembles) whose transmission 
takes place over independent and memoryless parallel channels. 
New upper bounds on the maximum-likelihood (ML) decoding er- 
ror probability are derived. The framework of the second version 
of the Duman and Salehi (DS2) bounds is generalized to the case 
of parallel channels, along with the derivation of optimized tilting 
measures. The connection between the generalized DS2 and the 
1961 Gallager bounds, known previously for a single channel, is 
revisited for the case of parallel channels. The new bounds are 
used to obtain improved inner bounds on the attainable channel 
regions under ML decoding. These improved bounds are applied 
to ensembles of turbo-like codes, focusing on repeat-accumulate 
codes and their recent variations. 

I. Introduction 

Performance analysis of linear block codes whose trans- 
mission takes place over parallel channels is of interest since 
communication over parallel channels models various practical 
scenarios. In this case the message is partitioned into several 
disjoint subsets, and the bits in each subset are transmitted 
over a different channel. This enables one to model transmis- 
sion over block-fading channels, rate-compatible puncturing 
of turbo-like codes, incremental redundancy retransmission 
schemes, cooperative coding, and multi-carrier signaling. 

Tight analytical bounds serve as a potent tool for assessing 
the performance of modern error-correction schemes, both for 
the case of finite block length and in the asymptotic case where 
the block length tends to infinity. For a single communication 
channel, these bounds can be applied in order to obtain a noise 
threshold which indicates the minimum channel conditions 
necessary for reliable communication. When generalizing to 
the scenario of independent parallel channels, this threshold is 
transformed into a multi-dimensional barrier in terms of the 
parallel-channel parameters, dividing the space into attainable 
and non-attainable channel regions. 

The performance of code ensembles operating under this 
setting was recently addressed by Liu et al. [6]; their analysis 
adapts the 1961 Gallager- Fano bounding technique [4] for 
communication over parallel channels. The upper bounds on 
the ML decoding error probability enable to derive achievable 
regions which ensure reliable communications under ML de- 
coding where the block length of the codes (or ensembles) tend 
to infinity; these regions depend on the asymptotic distance 




Fig. 1 . System model of parallel channels. 



spectrum of the ensemble of binary linear block codes under 
consideration (to be defined in Section ITU. 

Using a similar approach, we derive a parallel-channel 
generalization of the DS2 bound introduced in [3], [8]; for 
the case of parallel channels, we re-examine the well-known 
relations which exist for the single channel case between this 
bound and the 1961 Gallager bound (see [2], [8]). 

The new bounds are compared with the ones in [6] for 
parallel Gaussian channels, and show significant improvement 
over these bounds. They are also exemplified for various 
ensembles and provide tighter inner bounds on the boundary of 
the channel regions which are asymptotically attainable under 
ML decoding. The tightness of these bounds is exemplified 
for ensembles of accumulate-based codes. 

The remainder of the paper is organized as follows. The 
system model is presented in Section |ll| as well as prelim- 
inary material related to our discussion. In Section |lll| we 
derive the improved upper bounds under ML decoding when 
the transmission takes place over parallel MBIOS channels. 
Attainable channel regions for ensembles of codes are derived 
in Section ffvl Numerical results are given in Section fVl The 
reader is referred to the full paper version [9] where proofs 
and further mathematical details are provided. 

II. Preliminaries 

A. System Model 

The communication model consists of J statistically in- 
dependent, memoryless, binary-input and output-symmetric 
(MBIOS) channels, as shown in Fig. [2 Using a block code C 
of size M = 2 k , the encoder selects a codeword with equal 
probability (i) for transmission. Each codeword consists of 
n symbols, and the coding rate is defined as R = log ^ — = -| . 

The channel mapper selects for each coded symbol one 
of J channels through which it is transmitted; the transition 
probability of the j-th channel is p(y\x; j). The received vector 



is maximum-likelihood (ML) decoded at the receiver where 
the specific channel mapping is assumed to be known. 

Liu et al. [6] introduced the concept of a random channel 
mapping device which takes a symbol and assigns it to chan- 
nel j with probability a.j, and the assignment is independent 
of that of other symbols. This approach enables the derivation 
of an upper bound for the parallel channels which is averaged 
over all possible channel assignments and calculated in terms 
of the distance spectrum of the code (or ensemble). 

B. Distance Properties of Ensembles of Turbo-Like Codes 

Bounds on the ML decoding error probability of binary 
linear block codes or ensembles are often based on their 
distance properties (see, e.g., [8] and references therein). 

Let [C(n)] be an ensemble of codes of length n. We 
also consider a sequence of ensembles [C(ni)], [£(77,2)], ... all 
possessing a common rate R. For a given binary linear block 
code C, let A^ (or simply Ah) denote the distance spectrum, 
i.e., the number of codewords of Hamming weight h. Referring 
to ensembles of codes, denote by y^*-™-" the average distance 
spectrum of the ensemble. We are interested in studying the 
asymptotic case where n — > oo. To this end, we define the 
asymptotic exponent of the distance spectrum as 
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where 5 = — is the normalized distance. 
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III. Improved Bounds for Independent Parallel 
Channels under ML Decoding 

A. The DS2 Bound 

In this section, we present a generalization of the DS2 bound 
to the case of independent parallel MBIOS channels. For a 
discussion on the DS2 bound for a single MBIOS channel, 
the reader is referred to the tutorial paper [8, Chapter 4]. 

Theorem 1 [The Generalized DS2 Bound for Parallel Chan- 
nels]: Consider the transmission of binary linear block codes 
(or ensembles) under the scenario described in Section Hl-AI 
We have the following results: 

> The ML decoding error probability is upper-bounded by 
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and tp(-;j), j = 1, . . . , J are arbitrary probability tilting 
measures (which are subject to optimization, together 
with the parameters A and p). The bound in (0 is referred 
to as the DS2 bound for parallel channels. 
Consider the partitioning of the code C to constant 
Hamming-weight subcodes {Ch}f l= i where the subcode 
Ch consists of all the codewords in C of Hamming weight 
h plus the all-zero codeword. Applying the union bound 
over the subcodes yields 
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where P e \o(h) is the decoding error probability for the 
subcode Ch given that the all-zero codeword is trans- 
mitted. Rather than optimizing the tilting measure in 
the bound given by Eq. Q, we apply the DS2 bound 
for every subcode and use Eq. to evaluate the over- 
all bound on P e . The key result is that applying the 
bound in this manner allows us to optimize the J tilting 
measures for each subcode separately. The total number 
of subcodes does not exceed the block length of the 
code (or ensemble). Consequently, the use of the union 
bound does not degrade the related error exponent of 
the overall bound, but on the other hand, the optimized 
tilting measures are tailored for each of the constant- 
Hamming weight subcodes, a process which can only 
improve the exponential behavior of the resulting bound. 
The conditional ML decoding error probability of the 
constant-weight subcode Ch is therefore upper-bounded 
by 

hp 
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which can be written equivalently in the exponential form 

P e \o(h) < e -nEf\x, P j,{^}) 
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This exponential form will be useful in Section I1VI for 
the discussion on attainable channel regions. 
The optimized set of probability tilting measures 
{^(■ji)}/=i which attains the minimal value of the upper 



bound is given (for j = 1, . . . , J) by 
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The optimal parameters k and /3j j = 1 , . . . , J are related 
by the following implicit equations 
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In order to evaluate the bound in @ for a fixed pair of 
A and p, we find the optimized tilting measures in by 
first assuming an initial vector ((3\, . . . ,(3j) and then iterating 
between and l|8) until we get a fixed point for these 
equations. For a fixed S, we need to optimize numerically the 
bound in (0J w.r.t. the two parameters A and p. 

B. The 1961 Gallager Bound 

In this section we discuss the 1961 Gallager bound for 
parallel MBIOS channels. 

Theorem 2 [The 1961 Gallager Bound for Parallel Chan- 
nels]: Consider the transmission of binary linear block codes 
(or ensembles) under the scenario described in Section Hl-AI 
We have the following results: 

> The 1961 Gallager bound for parallel channels is given 
by (see [6]) 
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and f{-;j) is an arbitrary tilting measure, constrained to 
be non-negative and even (i.e., f(y;j) = f{—y,j))- 



By partitioning the code into constant Hamming-weight 
subcodes and using the union bound as in 0, we obtain 
optimized tilting measures for the 1961 Gallager bound. 
The optimal choice for the set of functions {f(-;j)} is 
given by 

f ( l- a (l- p -l) l-s(l-p-l) \ 2 
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where the parameters p, s, c are optimized numerically so 
as to get the tightest bounds within this form. 

Discussion. In [6], simple choices for these functions are 
used to evaluate the 1961 Gallager bound for parallel channels, 
rather than the optimized tilting measures. Consequently, the 
bounds in [6] are looser. 

The connection between the DS2 bound and the 1961 
Gallager bound for a single MBIOS channel has been explored 
in [2], [8]. In this case, it was demonstrated that the DS2 
bound is tighter than the Gallager bound. At first, one would 
expect that a similar result would also hold for the general 
case where the communication takes place over J independent 
parallel MBIOS channels. It is shown in [9] that this is 
not necessarily the case. Consequently, we cannot draw a 
conclusion establishing the superiority of one of these bounds 
over the other for the case where J > 1. We present numerical 
results which verify this observation. For technical details, the 
reader is referred to [9]. 

IV. Inner Bounds on Attainable Channel Regions 

A J-tuple of transition probabilities characterizing a par- 
allel channel is said to be an attainable channel point with 
respect to a code ensemble C if the average ML decoding 
error probability vanishes as we let the block length tend to 
infinity. The attainable channel region of an ensemble whose 
transmission takes place over parallel channels is defined as the 
closure of the set of attainable channel points. Since the exact 
decoding error probability under ML decoding is in general 
unknown, we evaluate inner bounds on the attainable channel 
regions whose calculation is based on upper bounds on the 
ML decoding error probability. 

Our numerical results referring to inner bounds on attainable 
channel regions are based on the following theorem. 

Theorem 3 [Inner bounds on the attainable channel re- 
gions for parallel channels]: Consider the transmission of a 
sequence of binary linear block codes (or ensembles) {[C(n)]} 
takes place over a set of J parallel MBIOS channels. Let 
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designate the Bhattachryya constants of the channels. Assume 
that the following conditions hold: 

1) 

inf E D&2 (5)>0, V<5 e(0,l) 

Sq<5<1 



where, for < S < 1, E DS2 (S) is calculated from l|5} 
by maximizing w.r.t. A, p (A > and < p < 1) and 
the probability tilting measures {V>( - ; i)}/=i- 



2) The inequality 
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is satisfied. 

3) There exists a sequence {£>„} of natural numbers tend- 
ing to infinity with increasing n so that 
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4) The normalized exponent of the distance spectrum 
r [c(«)] converges uniformly in S 6 [0, 1] to its asymptotic 
limit dT|. 

Then, the J-tuple vector of parameters characterizing these 
channels lies within the attainable channel region under ML 
decoding. 

Discussion. We note that conditions 3 and 4 in Theorem 3 
are similar to the last two conditions in [5, Theorem 2.3]. 
Condition 2 above happens to be a natural generalization of 
the second condition in [5, Theorem 2.3] to a set of parallel 
channels. The distinction between [5, Theorem 2.3] which 
relates to typical-pairs decoding over a single channel and 
the statement in Theorem 3 for ML decoding over a set of 
independent parallel channels lies mainly in the first condition 
of both theorems. 

A similar result which involves the generalized 1961 Gal- 
lager bound for parallel channels is given in a similar fashion 
by replacing the first condition with an equivalent relation 
involving the exponent of the 1961 Gallager bound maximized 
over its parameters, instead of the error exponent of the DS2 
bound. 

V. Performance Bounds for Turbo-Like 
Ensembles over Parallel Channels 

In this section, we exemplify the performance bounds 
derived in this paper for various ensembles of turbo-like 
codes whose transmission takes place over parallel Gaussian 
channels. We also compare the bounds to those introduced 
in [6], showing the superiority of the new bounds with their 
optimized tilting measures. 

A. Performance Bounds for Uniformly Interleaved Turbo 
Codes 

Fig. |2] compares upper bounds on the bit error probability 
of the ensemble of uniformly interleaved turbo codes of 



rate R 



bits per channel use. The encoder consists 



of two convolutional encoders with polynomials G(D) 



L 



1 + L> 4 



and the interleaver between them is 
of length 1000. The transmission is assumed to take place 
over two (independent) parallel binary-input AWGN channels 
where each bit is equally likely to be assigned to one of 
these channels (cti = o?2 = and the value of the energy 
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Fig. 2. Performance bounds for the bit error probability under ML decoding 
versus computer simulation results of iterative Log-MAP decoding (with 10 
iterations). 



per bit to spectral noise density of the first channel is set to 
(llo) = dB. The parallel-channel version of the union 
bound shown in the figure (see [9, Appendix C]) is obtained 
by averaging the union bound over all channel assignments, a 
similar process by which the improved bounds are obtained. 
The LMSF bound (yielding the tightest modified Shulman- 
Feder [10] bound) is a combination of the union bound with 
the Shulman-Feder bound (see [6], [9] for details on the 
derivation of this bound), and was presented in [6] as a 
special case of the 1961 Gallager bound. Clearly, the DS2 
and the 1961 Gallager bounds with their optimized tilting 
measures show a remarkable improvement in their tightness 
over the union and LMSF bounds in [6]; for a bit error 
probability of 10~ 4 , the improved bounds exhibit a gain of 
0.8 dB over the LMSF bound. The DS2 bound gains very 
little over the 1961 Gallager bound (about 0.05 dB) at a bit 
error probability of 10~ 3 . In spite of the remarkable advantage 
of the improved bounds over the union and LMSF bounds, 
computer simulations under (the sub-optimal) iterative Log- 
MAP decoding with 10 iterations show a gap of about 0.4 dB 
in favor of the performance under iterative decoding. This 
indicates that there is still room for further improvement in 
the tightness of the analytical bounds under ML decoding. 

B. Attainable Channel Regions for Various Ensembles of 
Accumulate-Based Codes 

In this section, we compare inner bounds on the attain- 
able channel regions of accumulate-based codes under ML 
decoding. The comparison refers to three ensembles of rate 
one-third, as depicted in Fig. |5J the first is the ensemble of 



uniformly interleaved and non-systematic repeat-accumulate 
(RA) codes with q = 3 repetitions; the second and the third 
are uniformly interleaved and systematic ensembles of RA 
codes and accumulate-repeat-accumulate [1] (ARA) codes, 
respectively, where the number of repetitions is equal to q = 6 
and, as a result of periodic puncturing, every third bit of the 
non-systematic part is transmitted (so the puncturing period is 
p = 3). For simplicity of notation, we make use of the abbre- 
viations NSRA(iV, q), SPRA{N,p, q) and SPARA(A r , M,p, q) 
for the encoders shown in Fig.[3](a)-(c), respectively (i.e., the 
abbreviations 'NS' and 'SP' stand for 'non-systematic' and 
'systematic and punctured', respectively). In this notation, N 
is the input block length. 

In [9], it is shown that the asymptotic growth rate of the dis- 
tance spectra of these ensembles is such that the second, third, 
and fourth conditions of Theorem 3 are satisfied. The distance 
spectra of the SPRA and SPARA ensembles considered here 
were calculated in [9] using the techniques introduced in [1]. 
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Fig. 4. Attainable channel regions for the rate 1/3 accumulate-based 
ensembles depicted in Fig.|3] The communication takes place over two parallel 
binary-input AWGN channels with ct\ = »2 = 0.5. The capacity limit and 
the attainable channel region which refers to the cutoff rate are referenced. 
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Fig. 3. Systematic and Non-systematic RA and ARA codes. 



The improved performance of the ensembles of SPARA 
codes under ML decoding is demonstrated by the Gallager 
bounding technique (combined with the optimization of the 
tilting measures) in Fig. |4] In this figure, a = denotes 
the fraction of bits which do not pass through the outer 
accumulator of the code. This enlargement of the attainable 
channel region for the ensemble of SPARA codes is attributed 
to the distance spectral thinning effect [7] which is more 
pronounced for this ensemble as compared to the other two 
(see [9]). This in turn yields an improved inner bound on the 
attainable channel regions, as observed in Fig.|4] It is shown in 
this figure that for the SPARA ensemble with the parameters 
P 



3, q = 6 and a 



15 



, the gap between the inner bound 



on the attainable channel region under ML decoding and the 
capacity limit is less than 0.05 dB. 
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